Repository navigation
perf(explore): rank full-text matches before reading their rows - #259
Merged
Merged
Conversation
searchNodesFTS sorted every match with all its node columns and joined through the id column of the external-content FTS table. It now ranks (rowid, score) in a subquery joined by rowid and reads full rows only for the page it returns. Ties still order by rowid. trezor-suite: FTS step 2.3-2.9x faster; explore round about 6% faster; output byte-identical.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #258. After it, the full-text search was the largest remaining explore statement.
searchNodesFTSjoinednodesonnodes_fts.id = nodes.idand sorted every match, with all its node columns, bybm25. In an external-content FTS tableidis read back fromnodes, and the join then looked it up again through theidindex. The query now ranks(rowid, score)in a subquery (still joined tonodesby rowid for the kind and language filters) and reads full rows only for the page it returns. Ties order by rowid, as the old form did implicitly.Measured on trezor-suite (151k nodes), recorded as ledger §5.100:
Checks: full suite 6,125 passed, 32 skipped;
readme-sync.README rows checked: the "About this fork" table and measured sections. No row quotes single-call explore latency, so none changed. The Bun concurrent-explore line was already marked for re-measurement in #258.